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OUR GOAL 

To assist educators and 
policymakers in their 
efforts to apply the 
evidence base to 
decisions about policies, 
programs, and practices 
they encounter. 



REQUEST: 

Indicators of success/dropout K-12: 

• Early warning system indicators that students are at risk of dropping out of high school (9th 
grade and throughout high school) 

• Early warning system indicators at the elementary and middle school level 

• Table showing information on states and districts in the Southeast (and beyond the region if 
identified during search) that have existing success factors/early warning databases 


RESPONSE 


K-12 research and literature on early warning and success indicators 

On-Track/Success Indicators Movement 

Over the past decade, research on dropout prevention has become focused on using evidence-based 
practice, and data-driven decisions, to mitigate students’ dropping out of high school and instead, 
support and prepare students for career and college. Early warning systems or on-track indicators, in 
which readily available student-level data are used to predict whether a student is likely to drop out, 
are being harnessed by schools, districts, states, and support organizations nationwide to help prevent 
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students from falling off the track to graduation and offer additional educational and support 
services to those who need them (Pinkus, 2008). Initiatives to create early warning systems are 
aligned with the What Works Clearinghouse Dropout Prevention Practice Guide 
recommendation that “utilizing data systems that support a realistic diagnosis of the number of 
students who dropout and that help identify individual students at high risk of dropping out” is a 
“critical first step” in effective intervention (Dynarski et al., 2008, p. 12). 

Indicator systems have been developed or adopted in Baltimore (Mac Iver, Plank, Durham, 
Farley- Ripple, & Balfanz, 2008), Boston (The Parthenon Group, 2007; Pinkus, 2008), Chicago 
(Allensworth & Easton, 2005, 2007), Dallas (Weir, 2008; Oakeley & Weir, 2010), Los Angeles 
(Lim & Pirone, 2007), Philadelphia (Neild & Balfanz, 2006; Neild, Balfanz, & Herzog, 2007), 
and Portland (Cielo & Leveen, 2007). Publications from the National High School Center 
provide guidance and tools to states, districts, and schools on constructing an early warning 
system that is effective for that district or school population, and recommend that an on-track 
indicator be considered for inclusion in such a system (Heppen & Therriault, 2008, p. 5). 

How are they constructed? 

To create indicator systems, states, school districts, and/or researchers have identified key 
student-level variables or a combination of key student-level variables (most commonly during 
grade 9) associated with high school graduation and dropping out of school. Most on-track or 
early warning systems use student-level administrative district and/or state data to construct 
indicators. Using this data, indicators are typically developed based on which variable(s) in the 
data best predict a student’s l ik elihood of graduating from high school. Therefore, successful 
early warning systems track multiple variables that are shown to be related to students’ 
likelihood of not graduating on time (Heppen & Therriault, 2008), such as poor grades in core 
subjects, low attendance, failure to be promoted, and disengagement in the classroom (Kennedy 
& Monrad, 2007). Sometimes, indicator systems use a single variable. However, typically the 
use of multiple variables increases the predictive yield (or accuracy) of the indicator. 

An on-track indicator is typically a "yes/no" variable indicating whether a student is "on-track = 
yes" or "not on-track = no" to graduate from high school. Students are categorized as either “on- 
track” or “off-track” for graduation based on this one indicator. Consequently, an on-track or 
success indicator is the inverse of an off-track or early warning indicator. 

The most commonly used student-level variables to make up an indicator are: 

• Attendance 

• Course failures in core courses (English, mathematics, social studies, and science) 

• Grades/credits earned in core courses 

• Disciplinary actions (e.g., suspensions) 

Less common, but noted in the literature are: 

• Transferring into, out of, or within the district 

• Withdrawing from school, returning to school 

• Standardized Test Scores 
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• Race/ethnicity 

• Gender 

• Poverty level/FRPL status 

• ESL or IEP categorization 

Benchmarks or a “red flag” for each of these variables, alone or in combination, is then 
established. For example, a threshold of 10% of instructional time missed during a school year 
might be indicated as a “red flag” for attendance. These benchmarks are usual empirically 
derived in that they are established based on the threshold that best distinguishes those students 
who drop out from those who graduate from high school. 

Which ones are most predictive of high school graduation? 

The most powerful predictors of whether a student will complete high school include course 
performance and attendance during the first year of high school (Allensworth & Easton, 2005; 
2007). Past research has shown that course-taking and school engagement behaviors in grade 9 
are "gateways" to advanced coursetaking in high school, which leads to high school graduation 
and postsecondary success. Studies have shown that graduation rates can be more highly 
correlated with such indicators than with standardized achievement test scores or student 
demographic characteristics (Allensworth & Easton 2005; Jerald 2006; Rumberger 2004). 

Most indicators have been developed during grade 9 because it is a critical transition period in 
the education pipeline (Heppen & Therriault, 2008) when the number of course failures and 
behavioral problems peak, and academic achievement declines (Hartman, Wilkins, Gregory, 
Gould, & D’Souza, 2011). However, because distress signals and patterns of behaviors emerge 
long before dropout occurs (Neild, Balfanz, & Herzog, 2007), there is increasingly a movement 
to create indicators for middle and elementary grades so that educators can early identify 
students at risk of not graduating. 

Researchers in Baltimore, Philadelphia, and Portland have examined indicators for students in 
grades 6-8. These studies have generally shown a similar pattern of findings with final grades in 
core courses, attendance patterns, academic achievement, and unsatisfactory behavioral marks 
predicting high school dropout (Maclver, 2011; Neild et al., 2007; Cielo & Leveen, 2007). In one 
study, student grade point average and class ranking as early as grade 4 predicted school dropout 
between grades 7 and 9 (Roderick, 1993). 

While the majority of indicator systems to date have been developed in large urban areas, there 
are increasingly efforts to develop indicators across urban, suburban, and rural populations. 
Evidence thus far suggests that similar indicators are useful at distinguishing students who do 
and do not graduate, but that the particular predictive yield or ability of specific indicators to 
differentiate students who do and do not graduate may vary based on the characteristics of the 
populations served (Hartman et al., 2011). For example, one study found that attendance was not 
as strong of a predictor as was found in prior studies in Philadelphia and Chicago (Johnson & 
Semmelroth, 2010). However, the authors comment that it may be because attendance data was 
not as reliably collected. In constructing an on-track indicator system it is therefore important to 
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consider both the characteristics of the populations served by specific districts or states as well as 
the reliability and availability of the data elements collected in those locales. 

Indeed, a key issue in developing early warning indicators is availability and quality of data used 
(Heppen & Therriault, 2008; Balfanz, Wang, & Byrnes, 2010). A systematic collection of 
student attendance and course performance data can be used to develop an effective early 
warning system that can also be tailored to local contexts. However, it should be noted that any 
indicator will, in practice, misidentify some students. This means that some students who are 
identified as on-track will fail to graduate on time, and that some students who are identified as 
off-track will graduate on time. The goal is to select an indicator that minimizes these 
misidentifications. 


Annotated Bibliography 

Indicators that students are at risk of dropping out of high school (9th grade and 

throughout high school) 

Allensworth, E.M., & Easton, J.Q. (2005). The ontrack indicator as a predictor of high 

school graduation. Chicago: University of Chicago, Consortium on Chicago School 
Research, http://ccsr.uchicago.edu/publications/p78.pdf 

CCSR website description: This indicator identifies students as ontrack if they earn at least five 
full-year course credits and no more than one semester F in a core course in their first year of 
high school. Ontrack students are more than three and one-half times more likely to graduate 
from high school in four years than offtrack students. The indicator is a more accurate predictor 
of graduation than students’ previous achievement test scores or then - background characteristics 

Perhaps the most important finding from this report is that failures during the first year of high 
school make a student much less likely to graduate. Based on their findings, the authors believe 
that parents and teachers should carefully monitor students’ grades, especially in the first 
semester of freshman year, when there are still many opportunities to improve grades. Helping 
students make a successful transition to high school during the first semester could make 
students more likely to graduate. 

This report also finds that ontrack students are not necessarily the students with the highest 
achievement test scores. Many students with strong achievement fail to graduate, and many 
students who have demonstrated weaker achievement succeed in graduating. 

Finally, this report concludes that the particular school a student attends plays a large role in 
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whether the student is on-track. While we expect schools to have students with differing levels of 
preparation for high school, differences in the number of students ontrack at each school 
remained even when the authors controlled for students’ eighth-grade test scores and 
socioeconomic status. This suggests that school climate and structure play a significant role in 
whether students succeed in high school. 

Schools can use the ontrack indicator, which makes use of readily available data on course 
credits and failures, to understand what aspects of the school may be leading students to drop 
out. 


Allensworth, E., & Easton, J. (2007). What matters for staying on-track and graduating in 

Chicago public high schools: A close look at course grades, failures and attendance in 

the freshman year. Chicago: Consortium on Chicago School Research. 

http ://ccsr. uchicago.edu/publications/07 % 20What % 20Matters % 20Final.pdf 

CCSR website description: This report found that freshman year course performance — more 
than background characteristics such as race, gender, socioeconomic status or prior 
achievement — predicts which students with disabilities are most at risk for dropping out of high 
school. 

The report, a joint effort of the Consortium on Chicago School Research and the National High 
School Center, found that absences, course failures, course credits, and GPA all can be used to 
accurately predict whether ninth-graders with disabilities will graduate from high school. 

Identifying these early warning indicators is especially crucial for students with disabilities, who 
drop out of high school at very high rates. Only 50 percent of CPS students with disabilities 
graduate in five years, compared with 70 percent of CPS students without disabilities. The 
national graduation rate of students with disabilities hovers around 57 percent, according to the 
U.S. Office of Special Education Programs. 

The new report builds on the findings of a 2007 CCSR report, What Matters for Staying On- 
Track and Graduating in Chicago Public High Schools, which showed that students who 
accumulate at least five semester-long credits and fail no more than one core course during their 
freshman year were nearly four times as likely to graduate. CCSR researchers incorporated 
those measures into an on-track indicator, adopted by CPS and other urban districts as part of 
their accountability systems, which proved more helpful in predicting graduation rates than 
either background characteristics or previous achievement. 
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The report released today demonstrates that this ontrack indicator can be applied to the special 
education population as well. Students with disabilities who are “ontrack” are three- to six-times 
more likely to graduate than their offtrack counterparts, according to the report. 

Most of the analyses in the report concern students who were first-time freshmen in 2004. 
Researchers separately analyzed five groups of students with different types of special needs: 
students with learning disabilities, mild cognitive disabilities, emotional disturbances, 
speech/language disabilities, 

and physical/sensory disabilities. In addition, researchers considered the course performance 
and academic behaviors of students who do not receive special education services but enter high 
school two or more years below grade level. 


Balfanz, R. (Jan 30, 2008). Three steps to building an early warning and intervention system 
for potential dropouts. (PowerPoint presentation). Baltimore, MD. Johns 
Hopkins University, Everyone Graduates Center. 

http://www.evervlgraduates.org/west-chris/item/95-three-steps-to-building-an- 

early-warning-and-intervention-svstem-for-potential-dropouts.html 

EGC website description: Most dropouts are identifiable years before they dropout, struggle in, 
or disengage from school for three to four or more years before they dropout. These dropouts are 
preventable, and ultimately they want to graduate from high school. 

This PowerPoint offers three steps toward reducing dropouts in your community. 

Step 1: Understand the dropout problem in your community. 

Step 2: Build an early warning, prevention and intervention system. 

Step 3: Involve the community. 


Balfanz, R., Bridgeland, J. M., Moore, L. A., & Fox , J.H. (2010). Building a grad nation: 
Progress and challenge in ending the high school dropout epidemic. Civic 
Enterprises; Everyone Graduates Center at Johns Hopkins University; America’s 
Promise Alliance, http://www.edweek.org/media/14grad civicmarshallplan.pdf 

Executive Summary: The central message of this report is that some states and school districts 
are raising their high school graduation rates with scalable solutions in our public schools, 
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showing the nation we can end the high school dropout crisis. America made progress not only 
in suburbs and towns but also in urban districts and in states across the South. 

Progress in states and school districts has often been the result of rising to a standard of 
excellence — with clear goals and expectations from the state to the classroom, by challenging all 
students with a more rigorous curriculum to obtain a meaningful diploma that prepares them for 
college and work, and through a targeted approach sustained over time that provides extra 
supports to the school leaders, teachers, and students who need them the most. Progress was not 
the result of a magic bullet but a weave of multiple reform efforts, sustained, integrated, and 
improved over time. 

Important progress is being made on a range of reforms, policies, and practices at all levels that 
will help ensure more students graduate from high school, ready for college and productive 
work. Although this is producing real results, including an increase in the national graduation 
rate, the pace is too slow to meet the national goal of a 90 percent high school graduation rate by 
2020. We must calibrate our educational system to the greater demands of the 21st century 
through a Civic Marshall Plan to make more accelerated progress in boosting student 
achievement, high school graduation rates, and college- and career-readiness for our nation to 
meet national goals and fulfill the promise of the next generation. 


Balfanz, R., & Byrnes, V. (2010). Dropout prevention through early warning indicators: A 
current distribution in West Virginia schools Baltimore: Johns Hopkins University 
Center for Social Organization of Schools. 

https://www.wvhepc.org/resources/Drop Out Prevention Report.pdf 

Authors’ abstract: This report provides a snapshot of the 2008-09 school year’s individual 
student level data for the 6th and 9th graders across the state of West Virginia. In particular, it 
focuses on indicators of students’ academic behaviors: attendance rates, behavioral problems 
(e.g., suspension, other disciplinary actions), and course marks (course failure or GPA). It then 
examines how 6th and 9th grade students exhibiting key behavioral warning signs that have been 
shown by research in other states and districts to be good predictors of future dropout are 
distributed across the state’s schools in concentration and number. The purpose of examining the 
distribution of students with these academic behaviors is to identify which schools most need 
additional resources in order to reduce the number of dropouts state- wide. In addition, we 
identify the high schools where the lowest proportions of 12th graders receive diplomas. Finally, 
the report examines the concentration of students with early warning indicators at the district 
level. The research in this report was funded by and conducted in conjunction with the National 
Governors Association Center for Best Practices dropout prevention and recovery state grants. 

Balfanz, R., Wang, A., & Byrnes, V. (2010). Early warning indicator analysis: Tennessee. 
Baltimore, MD. Johns Hopkins University, Everyone Graduates Center. 
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http://www.state.tn.us/education/safe schls/dropout/doc/FinalEarlyWarninglndicatorAnalysis.P 
DF 

Introduction excerpts: The purpose of this analytics project was to identify early warning 
indicators of dropout for the Tennessee Department of Education. Early indicators are “flags” or 
“distress signals” that are sent by a very large percentage of eventual dropouts years before they 
actually leave school. These “distress signals” indicate that a student is having trouble keeping 
up with schoolwork or is disengaging from schooling. . . . 

Early indicators of dropout are powerful tools at the K-12 level because they can potentially 
alert educators to students who need some level of intervention to stay on track to graduation. 
Identifying the relevant indicators is just a first step, and the step that is arguably the easiest. 
Indicators do no good if they are not followed up by action and further assessment to see whether 
the actions taken have helped to keep students ontrack to graduation. Nevertheless, appropriate 
action depends in part on a robust set of early warning indicators. A strong indicator set has 
several characteristics: 

1. They are empirically created. Powerful indicators are identified based on analysis of 
longitudinal data that tracks individual student progress over time. In essence, indicators use the 
experience of previous cohorts to intervene when students in subsequent cohorts begin to show 
behaviors associated with dropout among their elder siblings. 

2. They are simple and easily collected. Early warning indicators use readily available data that 
are typically maintained by schools — variables such as grades, attendance, and behavior in the 
classroom. They do not necessarily require complex statistical modeling techniques or access to 
data from surveys or interviews. 

3. The set of indicators has been refined to include a few key variables. In indicator systems, 
a few key indicators are easier for teachers to monitor than a large set of predictors. K-12 
analyses have demonstrated that although the underlying issues that produced the poor grade or 
weak attendance may be complex and may vary from student to student, there is a small number 
of flags that alerts educators to a student potentially falling offtrack. By extension, a good 
indicator system also identifies variables that are not the strongest predictors of eventual dropout. 

4. They capture the majority of students who eventually become dropouts. A good indicator 
system avoids the “1% problem,” or indicators that are highly predictive but only identify a small 
percentage of dropouts. 


Baltimore Education Research Consortium, (n.d.). On track and on time: Baltimore 
Education Research Consortium core analytic projects July 2008-June 2011. 
http://baltimore-berc.org/pdfs/On Track and On Time.pdf 
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Introduction excerpt: BERC joins City Schools in demonstrating a commitment to keeping on- 
level students on track to educational success (including an on-time high school graduation) and 
decreasing the dropout rate. To further these two goals, BERC will conduct eight projects to 
identify school and classroom practices that equip students for success at the elementary, middle 
grades, and high school levels. 

BERC research and analysis will address the following key research questions. What school and 
classroom practices equip students with the academic skills needed to stay ontrack and have 
successful transitions from first grade through middle school, high school, and postsecondary 
education? Why do many Baltimore students fall behind grade level, and what can teachers and 
principals do to help keep them on track? What school and classroom practices facilitate higher 
middle school and high school student achievement (with comparable incoming populations) 
among at-risk students who have shown early dropout indicators? What practices lead to higher 
levels of recovery and eventual on-time graduation? 

To answer these questions, BERC will access existing Baltimore City Public Schools data as 
well as conduct new research. Using a mixed-methods approach, BERC will pursue both 
quantitative analysis of longitudinal student- and school- level records (including aggregated 
teacher characteristics) and qualitative studies involving classroom observations (“shadowing” 
students across multiple classrooms and academic years to study effective instruction), principal 
interviews, teacher surveys, and focus groups. Samples of students and schools will intentionally 
include city schools that have proven themselves beacons of hope or excellence by 
demonstrating educational excellence multiple years in a row or student achievement 
significantly above other schools serving similar student populations and neighborhoods. 


Cielo, M.B., & Leveen, L. (2007). The fourth R: New research shows which academic 

indicators are the best predictors of high school graduation — and what interventions 
can help more kids graduate. Portland, OR: Connected by 25. 
www.conneetedby25.org/file download/46 

Article excerpts: This report summarizes the findings of research looking at data for the Portland 
Public Schools Class of 2004 as it moved through high school to expected graduation in June 
2004. The study was undertaken as the basis for determining how to implement support 
effectively to increase the number of students who graduate from high school. The research 
focused on learning what indicators best predict which students are at risk for failing to graduate 
and determining when, by year and quarter, students are most likely to disengage from school. 
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FINDING 1: Students are more likely to disengage from school at particular times of year and in 
particular years (a student’s likelihood of disengaging from school peaks during the summer and 
at the end of 12th grade). 

FINDING 2: Prior academic achievement (pre-high school) is an important indicator for 
predicting eventual graduation (students who, as 8th-graders, failed to meet 2 or more standards 
were 2.6 times more likely to leave school without graduating than their classmates in the 8 th 
grade who met 2 or 3 standards). 

FINDING 3: 9th grade is a pivotal year in which academic failure or success can 
be a powerful predictor of whether students will graduate (students who failed one or more core 
courses during 9th grade were almost 4 times more likely to leave high school before graduation 
than students who passed all their 9th- grade core courses). 

FINDING 4: Falling behind in course credits does not merely slow a student’s progress to 
graduation; it significantly increases the student’s risk of leaving PPS without graduating (A 
student with insufficient credits at the end of the 9th grade has 4. 1 times the risk of leaving 
school without graduating than a student with sufficient credits; a student with insufficient 
credits at the end of 10th grade has 5.2 times the risk; and a student with insufficient credits at 
the end of 1 1th grade has 5.6 times the risk). 

FINDING 5: Examining multiple academic indicators increases the accuracy with which a 
student’s l ik elihood of graduating can be predicted. 

FINDING 6: Many students experience one or more educational disruptions during high school 
(starting high school at age 15 or older; transferring into, out of, or within the district; or 
withdrawing from school and then returning), and these disruptions have different effects on a 
student’s lik elihood to graduate. (Students who were late arrivals to PPS were significantly less 
likely to graduate. These students left school without graduating at three times the rate of 
students who were already in the district or being classified as a “no show” in 8th grade or who 
entered the district in 9th grade.) 

When two key academic indicators — the number of 8th-grade standards met and the number of 
core courses passed or failed in 9th grade — are considered together, demographic variables such 
as race and poverty level cease to be statistically significant. 


Dynarski, M., Clarke, L., Cobb, B., Finn, J., Rumberger, R., and Smink, J. (2008). Dropout 
Prevention: A Practice Guide (NCEE 2008-4025). Washington, DC: National Center 
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for Education Evaluation and Regional Assistance, Institute of Education Sciences, 
U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc. 

Excerpt from the Introduction: This guide is intended to be useful to educators in high schools 
and middle schools, to superintendents and school boards, and to state policymakers in planning 
and executing dropout prevention strategies. The target audience includes school administrators 
as well as district-level administrators. This guide seeks to help them develop practice and policy 
alternatives for implementation. The guide includes specific recommendations and indicates the 
quality of the evidence that supports these recommendations. In addition, we have provided a 
description of some ways each recommendation could be carried out. Our examples should not 
be construed as the best or most effective ways to carry out each recommendation. Rather, the 
examples illustrate practices that were noted by previously implemented dropout prevention 
programs as having had an impact on staying in school, progressing in school, or completing 
school. Readers need to note that the specific ways in which the practices were implemented 
varied widely based on each school’s context. 


Gleason, P., & Dynarski, M. (2002). Do we know whom to serve? Issues in using risk 

factors to identify dropouts. Journal of Education for Students Placed at Risk, 7(1), 
25-41. 

Authors’ abstract: This article analyzes the effectiveness of widely used risk factors for 
identifying students who will drop out of school. The findings indicate that nearly all risk factors 
are not effective predictors of dropping out. The findings suggest that dropout prevention 
programs often serve students who would not have dropped out, and do not serve students who 
would have dropped out, which has implications for program effectiveness. [PDF file] 


Gwynne, J., Lesnick, J., Hart, H.M., & Allensworth, E.M. (2009). What matters for staying 
on-track and graduating in Chicago public schools: A focus on students with 
disabilities. Paper presented at the Consortium on Chicago School Research, 
Chicago. Available: http://eric.ed.gov/PDFS/ED507419.pdf . 

ERIC Abstract: A brief narrative description of the journal article, document, or resource. In this 
report, the authors look at the freshman year course performance of Chicago Public Schools 
(CPS) students who receive special education services and ask whether grades, course failures, 
absences, and on-track status are useful for identifying students who are at risk of dropping out. 
They also examine how academic behaviors, such as attendance and study habits, affect course 
failures and grades of students with disabilities. Consistent with the original "What Matters" 
report, most of the analyses in this report are based on information about the cohort of CPS 
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students who were first-time freshmen in 2004. When reporting graduation rates, the authors use 
information about the cohort of CPS students who were first-time freshmen in 2001. They report 
four major findings: (1) Students with speech/language disabilities and students with 
physical/sensory disabilities perform similarly to students without identified disabilities in their 
freshman year courses. Students with emotional disturbances and students who entered high 
school two or more years below grade level had the lowest level of course performance of any 
group they studied; (2) Freshman year course performance is a strong predictor of five-year 
graduation rates for students with disabilities and students who entered high school two or more 
years below grade level; (3) Higher absence rates are an important factor explaining why 
students with disabilities fail more classes and have lower grades than students without identified 
disabilities; and (4) Students with learning disabilities and students with mild cognitive 
disabilities do not benefit as much from rigorous study habits as students without identified 
disabilities. Appended are: (1) Study Samples; (2) Special Education Categories; (3) Statistical 
Models; and (4) Sample Attrition in Statistical Models. (Contains 6 tables, 20 figures, and 33 
endnotes.) 


Hartman, J., Wilkins, C., Gregory, L., Gould, L.F., & D’Souza, S. (2011 ). Applying an 
ontrack indicator for high school graduation: adapting the Consortium on Chicago 
School Research indicator for five Texas districts. (Issues & Answers Report, REL 
2011-No. 100). Washington, DC: U.S. Department of Education, Institute of 
Education Sciences, National Center for Education Evaluation and Regional 
Assistance, Regional Educational Laboratory Southwest. 
http://ies.ed.gov/ncee/edlabs/regions/southwest/pdf/REL 2011100.pdf 

Article summary excerpt: This study uses a measure of the ontrack or offtrack status of students 
at the end of grade 9 as an indicator of whether students in five Texas districts would graduate 
from high school in four years. In all five districts, on-time graduation rates were higher for 
students who were on track at the end of grade 9 than for students who were off track, both for 
students overall and for all racial/ethnic groups. 


Heppen, J., & Therriault, S.B. (2008). Developing early warning systems to identify 
potential high school dropouts. National High School Center. 
http://www.betterhighschools.org/pubs/documents/IssueBrief EarlyWarningSvstemsGuide 
.pdf 

NHSC website description: The information that follows and an accompanying tool developed 
by the National High School Center can help schools and districts to systematically collect early 
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warning indicator data so they can identify students at highest risk of dropout. An early warning 
system can be implemented at the school as well as at district levels. The role of the state is 
critical for providing support that can help districts and schools collect the key information with 
relative ease, including the use of integrated longitudinal data systems. 

This guide, intended for educators and policymakers at the school, district, and state levels, is 
designed to provide information about the following: 

• Factors that contribute to a student's dropping out 

• Research on early warning indicators 

• School- level early warning systems 

o Step-by-step instructions for how schools can calculate indicators and identify 
which students are on track to graduate and which are most likely to drop out 
while there is still time to intervene and prevent dropouts 

• District-level early warning systems 

o Information for districts regarding the development of district-wide early warning 
systems that begin with a local analysis of graduation and dropout patterns in the 
district 

• States' roles in supporting the development and use of early warning systems 


Jerald, C. (2006). Identifying potential dropouts: Key lessons for building an early warning 
data system [white paper]. Washington, DC: Achieve, Inc., American Diploma 
Project Network. http://www.achieve.org/files/FINAL-dropouts O.pdf . 

Introduction abstract: This white paper was prepared for Staying the Course: High Standards and 
Improved Graduation Rates, a joint project of Achieve and JFF funded by Carnegie Corp. of 
New York. Its goal is to provide policymakers with an overview of research about the dropout 
problem and the best strategies for building an early warning data system that can signal which 
students and schools are most in need of interventions. 


Johnson, E. (2008). REA fact sheet on the CPS freshman on-track graduation rate. Chicago, 
IL: Department of Applied Research, Chicago Public Schools. 
http://research.cps.kl2.il.us/export/sites/default/accountweb/Research/Fact Sheets/OnTrac 
k FactSheet.pdf 

This is a two-page FAQ-style summary of the Chicago Public Schools’ Freshmen On-Track rate. 
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Johnson, E., & Semmelroth, C. (2010). The predictive validity of the Early Warning 
System tool. NASSP Bulletin, 94(2), 120-34. 

Authors’ abstract: The Early Warning System is a tool developed by the National High School 
Center to collect data on indicators including attendance, grade point average, course failures, 
and credits earned. These indicators have been found to be highly predictive of a student's 
likelihood of dropping out of high school in large, urban areas. The Early Warning System tool 
was studied in two suburban schools. With the exception of attendance data, findings suggest 
that the indicators and suggested threshold for risk determination are predictive in suburban 
contexts. [PDF file] 


Kennelly, L., & Monrad, M. (2007). Approaches to dropout prevention: Heeding early 
warning signs with appropriate interventions. National High School Center. 

http://www.betterhighschools.org/docs/NHSC ApproachestoDropoutPrevention.pdf 

Executive Summary excerpts: There are effective, research-based steps that school systems can 
readily take to identify likely high school dropouts. Less is known about effective remedies 
designed to address dropout, though a variety of promising programs and interventions are 
available.... 

The key indicators that researchers have identified as indicative of who is most likely to drop out 
are 

• poor grades in core subjects, 

• low attendance, 

• failure to be promoted to the next grade, and 

• disengagement in the classroom, including behavioral problems. 

To be most effective in preventing dropout, school systems should focus dropout prevention 
efforts in the beginning of the middle grades. . . . 

School communities interested in building an early warning system should address the following 
steps: 

1. Establish a data system that tracks individual student attendance, grades, promotion status, and 
engagement indicators, such as behavioral marks, as early as fourth grade. 

2. Determine criteria for who is considered offtrack for graduation and establish a continuum of 
appropriate interventions. 

3. Track ninth grade students who miss 10 days or more of school in the first 30 days. The first 
month of high school provides important information about who is at risk of dropping out. Even 
moderate levels of absences are a cause for concern. Just one to two weeks of absence per 
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semester — which was typical for freshmen participating in a key Chicago study — was found to 
be associated with a substantially reduced probability of graduating. 

4. Monitor first quarter freshman grades, paying particular attention to failures in core academic 
subjects. Receiving more than one F in core academic subjects in ninth grade — together with 
failing to be promoted to tenth grade — is 85 percent successful in determining who will not 
graduate on time. Schools can offer immediate academic supports to the students who are failing 
in the first quarter of freshman year. 

5. Monitor Fall semester freshmen grades, paying particular attention to failures in core 
academic subjects. As first semester grades are posted, schools can develop individual student 
dropout strategies. By the end of the first semester, course grades and failure rates are slightly 
better predictors of graduation than attendance because they indicate whether students are 
making progress in their courses. 

6. Monitor end-of-year grades. The end-of-year grades will provide further information about 
failure rates and reveal grade point averages, providing detailed information about who is likely 
to struggle in later years and is considered by some researchers to be the best indicator for 
predicting nongraduates. In general, grades tend to be a more accurate predictor of dropout than 
test scores. 

7. Track students who have failed too many core subjects to be promoted to tenth grade. This 
provides perhaps the most critical information about which students should receive specialized 
attention and support. Research has shown that those who fail to be promoted are more likely to 
drop out. According to Alexander, Entwistle, and Horsey (1997), being held back trumps all for 
dropout indicators .... 

Currently, there is not an extensive menu of proven strategies and interventions tailored for key 
dropout prevention initiatives most appropriate for various risk factors at differing stages across 
the education pipeline. However, there are a few proven dropout prevention programs featuring 
key components, such as 

• attendance and behavior monitors, 

• tutoring and counseling, 

• establishment of small learning communities for greater personalization, 

• engaging catch-up courses, 

• Ninth Grade Academies, 

• homerooms, 

• benchmarking, 

• progress monitoring, 

• tiered interventions, 

• a focus on equal access to rigorous coursework and high expectations, 

• career/college awareness, 

• community engagement, and 
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eighth-to-ninth grade transition programs. 


Some of the common elements shared across numerous programs include attention to school 
climate in order to facilitate student engagement, rigorous coursework for all students, and the 
effective use of extended learning time during the school day such as the block schedule.... 

A lot still is not known about dropout prevention strategies and interventions that make a 
positive difference. However, interventions that have the capacity to be oriented around 
individual student needs, and that work in tandem with schoolwide interventions able to adjust 
around grade-level needs, hold promise as an effective combination for combating the nation’s 
dropout problem. 


Lim, C., & Pirone, J. (2007). Using data for dropout prevention: Can we identify at-risk 

students before it’s too late? Planning and Assessment, Los Angeles Unified School 
District. 

http://www.mvfuturemvdecision.Org/ourpages/auto/2008/5/l/1209652805638/Data%20Driv 
en % 20Decision % 20Making.ppt#l 


This 32-slide PowerPoint presentation discusses the authors’ use of indicators from Chicago and 
Philadelphia research to examine a cohort of LAUSD students who entered 9 th grade for the first 
in 2003. It traced them back to middle school (looking at attendance, math and ELA grades, and 
GPA) and forward to high school (looking at graduation/dropout/transfer rates, GPA, and credit 
accumulation). The authors identified their own at-risk indicators: 

• Cumulative failures in ELA and/or math in middle school 

• Cumulative attendance rates in middle school 

• Credit accumulation after the first year of high school 

The presentation also discusses systems already in place to identify at-risk students and discusses 
LAUSD’ s “Decision Support System.” 


Mac Iver, M.A. (2010). Gradual disengagement: A portrait of the 2008-09 dropouts in the 
Baltimore City Schools. Baltimore, MD: The Baltimore Education Research 
Consortium. 

http ://baltimore-berc.org/pdfs/Gradual % 20Disengagement % 20final.pdf 

Author’s abstract: Gradual Disengagement: A Portrait of the 2008-09 Dropouts in the 
Baltimore City Schools examined eight years of student-level data to paint a collective portrait of 
the process of disengagement that leads to student dropout. The study found that the majority of 
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students who eventually drop out of Baltimore high schools enter ninth grade with a pattern of 
chronic absenteeism that goes back at least several years. A large majority of eventual dropouts 
are overage for grade by the time they enter ninth grade for the first time, and have increasingly 
high levels of absenteeism and course failure over their years in high school. The study 
recommends that interventions be implemented during the early middle grades to prevent most 
dropout outcomes, and that non-traditional credit-earning options be offered to older enrolled 
students (17 and older) who already have patterns of chronic absenteeism and course failure. 


Mac Iver, M.A., & Mac Iver, D.J. (2009). Beyond the indicators: An integrated school-level 
approach to dropout prevention. Arlington, VA: The Mid- Atlantic Equity Center, 
The George Washington University Center for Equity and Excellence in Education. 
http://diplomasnow.files.wordpress.com/2011/02/dropout-report-8-ll-09.pdf 

Executive Summary excerpt: This report summarizes the research on why students drop out of 
school, explains the research implications for how to create an integrated dropout prevention 
strategy, and highlights an innovative pilot project that yielded results in a matter of months — a 
how-to example that works. 


Mac Iver, M.A., & Mac Iver, D.J. (2010). Keeping on track in ninth grade and beyond: 
Baltimore’s ninth graders in 2007-08. A Research Report by the Baltimore 
Education Research Consortium. 

http://baltimore-berc.org/pdfs/3A %20Final%20report 06-15-10.pdf . 

Executive Summary excerpts: Raising the graduation rate in Baltimore City will require specific 
attention to addressing the behavioral factors identified in previous research that push students 
offtrack to graduation, particularly chronic absenteeism, suspensions, and course failure in ninth 
grade. Researchers further hypothesize that interventions to reduce the incidence of these 
behaviors associated with non-graduation will help to increase graduation rates, though such 
intervention studies have not yet been underway long enough for graduation outcomes to be 
measured. Such interventions will often need to address underlying causes of behavioral 
indicators. A first step, prior to organizing intervention strategies and evaluating their 
effectiveness, is to describe the extent and concentration of these ninth grade early warning 
indicators, which is the primary goal of this study. 

Analysis of Baltimore City Schools data for the 6,662 first-time ninth graders in 2007-08 
indicated that: 
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• Chronic absenteeism was widespread. 

• Core course failure was even more common than chronic absenteeism. 

• Suspensions were much less prevalent. 

Implications of the Findings 

Raising the graduation rate in Baltimore City will particularly require specifically targeted efforts 
to increase attendance and reduce ninth grade course failure. The large number of students 
exhibiting these warning signals demands extensive district support to those schools where 
concentrations are extremely high. Efforts to increase attendance must begin much earlier than 
high school, since most of those chronically absent in ninth grade had poor attendance patterns 
already established in prior years. 

The district’s Master Plan already includes numerous action steps designed to increase 
attendance. While there is considerable discussion underway of the steps being taken, it is 
important for the district to consider a more formal analysis of ongoing efforts to increase 
attendance. Similarly, while the Master Plan also notes the need to address course failure, it is 
crucial to collect systematic data on what schools are actually doing to prevent course failure as 
well as to provide credit recovery options for students who need them. This is particularly 
important given the significant additional costs associated with credit recovery (for nearly 27,000 
core courses failed at Baltimore City high schools grades 9-12 in a single academic year). While 
some course failure is directly linked to students' irregular attendance, other course failure seems 
to have different roots. 

BERC is currently planning a study to analyze school and classroom practices associated with 
higher levels of ninth grade course passing in Baltimore’s schools. An additional BERC study of 
the district’s efforts to increase attendance would provide useful information for future data- 
driven decisionmaking. We believe that formal analysis of the current efforts underway to 
address attendance and course failure in particular will also help district leaders to better 
integrate multiple programs and strategies into a more systematic framework for dropout 
prevention. Implementation of early warning systems and public health-style tiered prevention 
models designed to keep students from falling off-track to graduation (particularly in terms of 
credits accrued) will be an important step to increase the district’s graduation rate. 


Mac Iver, M., Plank, S.B., Durham, R., Farley- Ripple, E., & Balfanz, R. (2008). The 
challenge of on-time arrival: The seven-year flight paths of Baltimore’s sixth 
graders of 1999-2000. Research Report. Baltimore, MD: Johns Hopkins University. 
http://www.baltimore-berc.org/pdfs/SIXTH%20pathwavs5-13-08.pdf 
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Executive Summary: This report explores the pathways followed by students who were sixth 
graders in the Baltimore City Public School System (BCPSS) in 1999-00. By following this 
cohort of sixth-grade students over the next seven years, through their expected (on-time) 
graduation year (2005-06), we learn much about patterns of promotion and retention, mobility 
within BCPSS, transfer out of BCPSS, and graduation rates. In particular: 

• A total of 8,560 students were enrolled in sixth grade in BCPSS in 1999-00. Most of these 
(85.4%) qualified for free or reduced-price lunch, and more than one in five (21.2%) had special 
education status. About one in twelve (8.4%) was repeating sixth grade in 1999-00. Five percent 
were new to BCPSS in that year. On average, this cohort scored at the 19th percentile in reading 
and the 22nd percentile in mathematics in sixth grade. Nearly three in four attended a large 
middle school. 

• This cohort began elementary school before the implementation of districtwide curricular 
reforms in 1998. The creation of more K-8 schools occurred after these students began the 
middle grades, and they entered high school before the opportunity to attend smaller, innovation 
high schools existed for more than a small percentage. This cohort study, therefore, establishes a 
baseline of student outcomes before the effects of district reforms over the past decade could be 
expected. 

• There was a steady decline year-by-year in the percentage of students enrolled for at least part 
of the year within BCPSS, with this decline accelerating during the later high school years. At 
the beginning of 2005-06, just 55.9 percent were still enrolled in the district. 

• By the end of 2005-06, more than half of the cohort (54.4%) had left the district. About one in 
five (19.4%) was a documented transfer to another district or private school; most of the rest had 
either dropout codes or missing data in the withdrawal code. One in three (33.6%) had graduated 
from the district, and the rest were still in school. 

• Though three-quarters of those still enrolled in the district were ontrack in ninth grade (2002- 
03), nearly one in five was not promoted the following year. The average ninth-grade attendance 
rate of students who were retained in ninth grade after 2002-03 was significantly lower than that 
of those who were promoted after ninth grade, and this rate appeared to be the culmination of a 
declining pattern of attendance over the previous three years. 

• High rates of mobility during the middle grades were associated with lower rates of on-time 
graduation. Students who attended the same school for sixth grade and the two years following 
were much more likely to graduate on time (52.2%) than were those students who were still 
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enrolled in BCPSS in 2001-02, but had changed schools at least once (27.2% on-time graduation 
rate). 

• Chronic absenteeism was the norm for this cohort. A third of the sixth graders in 1999-00 
missed at least one-ninth of their days on roll (20 days of a full school year). This percentage 
increased over time through 2003-04, when 56.4 percent of students were chronically absent. 
The rate declined somewhat the two subsequent years, undoubtedly because many of these 
chronically absent students were no longer enrolled in BCPSS. 

Chronic absenteeism and student academic struggles in the middle grades are particularly crucial 
issues to address, given their association with the failure to arrive at on-time graduation. At the 
same time, another important finding of this report is that many BCPSS students who do manage 
to successfully graduate on time have also manifested chronic absenteeism and academic 
struggles (low test scores) that will almost certainly have an influence on their performance in 
postsecondary education and the workplace. Missed opportunities to learn, as well as more 
general patterns of absenteeism, have weakened the core foundation of student preparation for 
the world beyond high school. Increasing student engagement with learning opportunities at the 
secondary school level remains the foremost challenge for the Baltimore City Public School 
System in the coming years. 


Neild, R.C. (2009). Falling off track during the transition to high school: What we know 
and what can be done. The Future of Children, 19( 1), 53-76. 

Summary: Ninth grade, observes Ruth Curran Neild, marks a critical juncture in American 
schooling. Students who manage the academic demands of the transition to high school have a 
high probability of graduating four years later. But those who do not — who fail to earn as many 
credits as they should during ninth grade — face a substantially elevated risk of dropping out of 
high school. Neild examines four theories about why ninth grade poses difficulties for some 
students. The first is that ninth grade coincides with life-course changes, such as reduced parental 
supervision and increased peer influence. The second is that in moving to a new school, students 
must break the bonds they have formed with their middle-school teachers and peers. The third is 
that some students are inadequately prepared for high school. The final theory is that the 
organization of some high schools is itself a major source of students' difficulty. Each theory, 
says Neild, suggests a particular type of policy response. 

The strongest evidence, observes Neild, points to inadequate preparation for high school and the 
organization of high schools. Reform efforts thus far have tended to address high school 
organization, with or without a focus on instructional quality or helping students to catch up on 
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academic skills. Evaluations of these reforms, says Neild, suggest that both school organization 
and instructional improvement are necessary to keep ninth graders on track to graduation. 

Neild notes that school districts and state departments of education also are addressing the 
problem. In addition to supporting comprehensive school reform with a focus on ninth graders, 
districts have created accountability indicators of how well high schools are keeping ninth 
graders on track. States are helping districts to develop their capacity to maintain and analyze 
data on ninth-grade progress, including "early warning indicator systems" that identify students 
who are falling off track to graduation. [PDF file] 


Neild, R., Balfanz, R., & Herzog, L. (2007). An early warning system. Educational 
Leadership, 65(2), 28-33. [PDF file] 

This article provides an overview of research findings from Philadelphia, Boston, and 
Indianapolis. It discusses the national dropout problem; early warning indicators (for 6 th grade: 
failing math; failing English; attendance less than 80%; and an “unsatisfactory” behavior grade 
in at least one class); the difficulty of the 9 th grade transition; and interventions for at-risk middle 
school students, struggling ninth graders, and students who have already dropped out of school. 


Parthenon Group. (2007). Strategic planning to serve off-track youth: Data review and 

strategic implications. (PowerPoint presentation) Boston, MA: Parthenon Group. 
http ://8. 12.35.242/files/Parthenon %20complete % 20report.pdf 

This 37-slide PowerPoint presentation provides a look at Boston Public Schools’ (BPS) efforts to 
identify and assist off track students. It identifies the BPS dropout problem and uses BPS data to 
create a list of four predictive and comprehensive risk factors: 

• ELL students entering BPS for the first time in high school 

• All students who are substantially separate SPED at any point in grades 9-12 

• Students with one or more 8 th grade risk factor (attendance below 80%, two or more 
years overage, fail two or more core courses) 

• Students failing multiple core courses (English, Math, Science, or History) in 9 th grade 

It also identifies each BPS high school’s percentage of at-risk students, discusses the district’s 
“First Chance” intervention system and provides funding and results by high school, and assesses 
the poor performance of district alternative education schools. 
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Pinkus, L. (2008). Using early-warning data to improve graduation rates: Closing cracks in 
the education system. Washington, DC: Alliance for Excellent Education. 
http://www.all4ed.org/files/EWI.pdf 

Article excerpt: This brief explores the predictive power of early-warning data, offers examples 
of current efforts to use such data to guide secondary school interventions across the country, and 
discusses the policies that can support these efforts. 


Ponder, P. (n.d.) Graduation pathways: Using freshman year indicators. Doing What Works. 

http://dww.ed.gov/Dropout-Prevention/Rigorous-Relevant- 
Instruction/see/index.cfm?T ID=24&P ID=53&cl=1254 


DWW website description: This PowerPoint presentation outlines Chicago Public Schools’ 
(CPS) districtwide effort to prepare all students for postsecondary and employment success. This 
initiative focuses on building the capacity of principals and Instructional Leadership Teams 
(ILT) to analyze and use data, set strategic goals, employ targeted strategies, and continuously 
monitor progress. A key driver of this process is an intensive focus on freshman year ontrack 
indicators — e.g., attendance and middle school academic performance. The district’s 
Instructional Leadership Council (ILC) and the ILTs work together to use the district’s Early 
Warning Reporting System to develop a Freshman Watch List, a Freshman Success Report, and 
a Credit Recovery Report for schools. Based on these reports, schools can take immediate action 
to plan strategies and interventions to keep students ontrack to graduation. Other districts may 
find this presentation useful as they develop similar initiatives that focus on using data such as 
the freshman ontrack indicators. 


Rumberger, R.W., & Arellano, B. (2007). Student and school predictors of high school 
graduation in California. Santa Barbara: California Dropout Research Project, 
University of California. 

Authors’ abstract: Solving the dropout crisis in California requires a better understanding of the 
nature and causes of the problem. This report analyzes student and school predictors of high 
school graduation based on a sample of 1,343 California tenth grade students who attended 63 
public high schools in 2002. The analysis is based on survey data collected in 2002 from 
students, teachers, principals, and parents, and transcripts collected one year after students' 
expected graduation in 2004. This study identified a number of alterable student and school 
predictors of high school graduation, including student engagement and achievement as well as 
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school academic climate. In all, the findings from this study support the argument that solving 
California's high school dropout crisis will take a multifaceted approach — it will require better 
preparing students before they enter high school, addressing their social as well as their 
academic needs while in high school, and it will require improving high schools themselves. 
[PDF file] 


Silver, D., Saunders, M., & Zarate, E. (2008). What factors predict high school graduation 
in the Los Angeles Unified School District ? Santa Barbara: California Dropout 
Research Project. 

Authors’ abstract: Because the paths to high school graduation or to dropping out begin years 
before these events, identifying relevant school-related factors requires a comprehensive 
analysis of data at the district, school, and student levels. In collaboration with the Los Angeles 
Unified School District (LAUSD), the authors of this study analyzed district data to track the 
educational progress of all first-time 2001-02 9th graders, from the 6th grade through to their 
expected graduation in the spring of 2005. This group consisted of 48,561 students who 
attended 163 LAUSD middle and high schools. The analysis of transcript records, standardized 
test scores, and a broad database of student and school characteristics sheds light on the middle 
and high school factors related to high school persistence and graduation. 

The study exposes troubhng rates of academic failure, but it also offers reasons for hope, 
demonstrating that academic experiences and school factors play a much larger role than 
student demographics in determining graduation rates, and that there is tremendous variation in 
the extent to which schools can have success with populations of students whose odds of 
graduation are typically quite poor. [PDF file] 


Temple, J., Reynolds, A., & Miedel, W. (2000). Can early intervention prevent high school 
dropout? Evidence from the Chicago Child-Parent Centers. Urban Education, 35(1), 


31-56. 


Authors’ abstract: The authors investigated the effects of participation in the Chicago Child- 
Parent Center and Expansion Program from ages 3 to 9 on school dropout by age 18 for 1,159 
youths. This Title I program provides child education and family support services from preschool 
through second or third grade in 20 sites in Chicago’s poorest neighborhoods. After comparing 
children in 20 program sites with children who attended schools in similarly poor neighborhoods 
without the intervention, we found that preschool participation was associated with a 24% 
reduction in the rate of school dropout and that participation for 5 or 6 years was associated with 
a 27% reduction in the rate of early school dropout relative to less extensive participation. [PDF 
file] 
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Texas Education Agency. (2001). How to use the 2003 Early Indicator Reports. 
http://ritter.tea.state.tx.us/taa/perfreport010309how.doc 

Introduction: This document uses results from the state 2003 Early Indicator Report, Part I, to 
illustrate analyses that educators may wish to reproduce with local results. In each case, an 
example expectation is defined and then evaluated. These example expectations are for 
illustration only; they are not accountability rating standards. As these early indicator reports are 
designed to focus on preparing students to meet the new exit-level testing requirement, the 
ultimate goal is for all students to be eligible to graduate. 


Weir, D. (2008). Development of the Dallas ISD on-track indicator, 2007-08. 

http://www.dallasisd.org/eval/evaluation/atglance2009/EA08-153-4-At-a-Glance-Qn-Track- 

Indicator.pdf 
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Indicators at the elementary and middle school level 


Alexander, K.L., Entwistle, D.R., & Horsey, C. (1997). From first grade forward: Early 
foundations of high school dropout. Sociology of Education, 70(2), 87-107. 

Authors’ abstract: In tracking the educational progress of a sample of Baltimore school-children 
from entrance into first grade in fall 1982 through early spring 1996, the authors examined the 
children's personal qualities, first-grade experiences, and family circumstances as precursors to 
high school dropout. Logistic regression analyses were used to identify predictors of dropout 
involving family context measures (stressful family changes, parents' attitudes, and parents' 
socialization practices), children's personal resources (attitudes and behaviors), and school 
experiences (test scores, marks, and track placements). These various measures were found to 
influence dropout independently of sociodemographic factors and account for much of the 
difference in the odds of dropout associated with family socioeconomic status, gender, family 
type, and other "risk factors." The authors take a life-course perspective on dropout, viewing it as 
the culmination of a long-term process of academic disengagement. [PDF file] 


Balfanz, R. (2008). Early warning and intervention systems: Promise and challenges for 
policy and practice. Prepared for National Academy of Education and National 
Research Council Workshop on Improved Measurement of High School Dropout 
and Completion Rates. 

http ://www7.nationalacademies.org/BOTA/Paper % 20bv % 20R. % 20Balfanz.pdf 

Author’s abstract: This paper reviews the work we have done to date on early warning and 
intervention systems, primarily in the middle grades. It then draws from our findings and 
experience to illuminate some critical issues and challenges states and school districts will need 
to address in order to be able to use on and off track indicators for high school graduation in 
effective and productive manners. The first section examines using off track indicators to build 
early warning systems. The second section looks at using early warning systems to support 
successful interventions to keep students on the path to high school graduation. The final section 
looks at how off track indicators might be used to identify which middle and high schools face 
the greatest educational challenges, distribute resources based on the degree of educational 
difficulty, and evaluate the relative success of schools in meeting their educational challenges. 


Balfanz, R. (2009). Putting middle grades students on the graduation path. A policy and 
practice brief. Everyone Graduates Center and Talent Development Middle 
Grades Program. 
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http://www.nmsa.Org/portals/0/pdiyresearch/Research from the Field/Policy Brief Balfan 
z.pdf 


Introduction excerpt: This brief, drawing on our research and field work, illuminates key policy 
and practice implications of the middle grades playing a stronger role in achieving our national 
goal of graduating all students from high school prepared for college or career and civic life. The 
brief is based on more than a decade of research and development work at the Center for the 
Social Organization of Schools at Johns Hopkins University. It also draws on direct field 
experience in more than 30 middle schools implementing comprehensive reform and a 
longstanding collaboration with the Philadelphia Education Fund. 


Balfanz, R., & Byrnes, V. (2010). Dropout prevention through early warning indicators: A 

current distribution in West Virginia schools Baltimore: Johns Hopkins University 

Center for Social Organization of Schools. 

https://www.wvhepc.org/resources/Drop Out Prevention Report.pdf 

Authors’ abstract: This report provides a snapshot of the 2008-09 school year’s individual 
student-level data for the 6th and 9th graders across the state of West Virginia. In particular, it 
focuses on indicators of students" academic behaviors: attendance rates, behavioral problems 
(e.g., suspension, other disciplinary actions), and course marks (course failure or GPA). It then 
examines how 6th and 9th grade students exhibiting key behavioral warning signs that have been 
shown by research in other states and districts to be good predictors of future dropout are 
distributed across the state’s schools in concentration and number. The purpose of examining the 
distribution of students with these academic behaviors is to identify which schools most need 
additional resources in order to reduce the number of dropouts statewide. In addition, we identify 
the high schools where the lowest proportions of 12th graders receive diplomas. Finally, the 
report examines the concentration of students with early warning indicators at the district level. 
The research in this report was funded by and conducted in conjunction with the National 
Governors Association Center for Best Practices dropout prevention and recovery state grants, 
[also in high school indicators section] 


Balfanz, R., & Herzog, L. (2006). Keeping middle grades students on track to graduation: 
Initial analysis and implications. (PowerPoint presentation) Johns Hopkins 
University and Philadelphia Education Fund. 

http://philaedfund.org/sites/default/liles/media/dropoutresearch 4.06.pdf 
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This 40-slide PowerPoint presentation summarizes research in Baltimore and Philadelphia. The 
authors identify four 6 th grade predictors of falling off track to graduation: attending school less 
than 80% of the time; poor behavior; failing math; and failing English. Poverty and adolescence 
increase the l ik elihood of having a risk factor. They suggest three levels of intervention — school- 
wide, targeted, and intensive — and provide numerous examples of each. 


Balfanz, R., Herzog, L., & Mac Iver, D.J. (2007). Preventing student disengagement and 
keeping students on the graduation path in urban middle-grades schools: Early 
identification and effective interventions. Educational Psychologist, 42(4), 223-235. 

Authors’ abstract: This article considers the practical, conceptual, and empirical foundations of 
an early identification and intervention system for middle-grades schools to combat student 
disengagement and increase graduation rates in our nation's cities. Many students in urban 
schools become disengaged at the start of the middle grades, which greatly reduces the odds that 
they will eventually graduate. We use longitudinal analyses — following almost 13,000 students 
from 1996 until 2004 — to demonstrate how four predictive indicators reflecting poor attendance, 
misbehavior, and course failures in sixth grade can be used to identify 60% of the students who 
will not graduate from high school. Fortunately, by combining effective whole-school reforms 
with attendance, behavioral, and extra-help interventions, graduation rates can be substantially 
increased. [PDF file] 


Baltimore Education Research Consortium. (2011). Destination graduation: Sixth grade 
early warning indicators for Baltimore City Schools. Their prevalence and impact. 
Baltimore, MD: Author. 

http://baltimore-berc.org/pdfs/SixthGradeEWIFullReport.pdf 

BERC website description: A new report from the Baltimore Education Research Consortium 
has examined the 2000-01 cohort of sixth graders from the Baltimore City Schools in search of 
reliable indicators of eventual dropout that would justify intervention. The report found four 
prevalent early warning indicators of non- graduation: chronic absence (defined as missing 20 or 
more days of school); failing English, math, or both, and/or a failing average of English, math, 
science, and social studies; being at least one year over-age for grade; and being suspended for 
three or more days. Overall, researchers found that only a third of students with at least one early 
warning indicator in the sixth grade went on to graduate within one year of expected graduation. 
Students with no warning indicators graduated at more than double the rate. However, looking at 
more recent cohorts, researchers found that the prevalence of some indicators have decreased. In 
comparison with sixth graders in 2000-01, those in 2008-09 had substantially less chronic 
absence, course failure, and suspension of three days or longer. Given the impact of retention on 
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graduation outcomes, the report recommends the district continue to monitor retention policies 
and practices. 


Durham, R.E., & Plank, S.B. (2010). Maintaining high achievement in Baltimore: An 

overview of the elementary grade trajectories of four recent city schools first grade 
cohorts. Baltimore, MD: Baltimore Education Research Consortium. 
http://baltimore-berc.org/pdfs/lA%20Issue%20Brief FINAL 06-21-10.pdf 

BERC website description: To address success at the elementary school level, BERC has 
produced the following report, Maintaining High Achievement in Baltimore , which examines the 
performance of four successive first grade cohorts spanning the years 1999-00 through 2008-09. 
The results of this study show that City Schools students are scoring Proficient and Advanced on 
the Maryland School Assessment with increasing frequency. Further, improvements in City 
Schools student achievement in grades three to five represent a faster rate of acceleration than 
what has occurred among Maryland students as a whole. The study explores the relationships 
between students’ initial first grade achievement and subsequent performance in reading and 
math in grades three, four, and five. The study also addresses patterns of out-migration from City 
Schools, and acknowledges the changing educational policy environment in Baltimore over the 
past decade. 


Gleason, P., & Dynarski, M. (2002). Do we know whom to serve? Issues in using risk 

factors to identify dropouts. Journal of Education for Students Placed at Risk, 7(1), 
25-41. 

Authors’ abstract: This article analyzes the effectiveness of widely used risk factors for 
identifying students who will drop out of school. The findings indicate that nearly all risk factors 
are not effective predictors of dropping out. The findings suggest that dropout prevention 
programs often serve students who would not have dropped out, and do not serve students who 
would have dropped out, which has implications for program effectiveness. [PDF file] [also in 
high school section] 


Herzog, L. (2009, Spring). Two schools use ‘early warning system’ to avert dropouts. 

Philadelphia Public School Notebook, 16(3), 26-27. 
http ://www. thenotebook.org/sp ring-2009/091091/two-schools-use- % E2 % 80 % 98early- 
warning % E2 % 80 % 99-svstem-avert-dropouts 
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This brief article describes the Philadelphia Public Schools’ Keeping Middle Grades Students 
On-Track to Graduation: The Early Warning Indicators Project. The project applies the four 
early warning indicators identified by Johns Hopkins researchers — poor attendance, repeated 
behavior problems, failing English, and failing math — to target 6 th -8 th grade students in need of 
additional support, then uses City Year and Community in Schools workers to provide the 
interventions to keep the students on track for graduation. 


Kurlaender, M., Reardon, S., & Jackson, J. (2008). Middle school predictors of high school 
achievement in three California school districts. Santa Barbara: California 
Dropout Research Project, http://cdrp.ucsb.edu/dropouts/pubs reports.htm . 

Authors’ abstract: This paper explores early predictors of high school graduation and success. 
Employing 7th grade cohorts from three large California school districts (San Francisco, 

Fresno, and Long Beach), we investigate the role of several key middle school academic 
performance measures in identifying diploma receipt, passing the California High School Exit 
Examination on the first attempt, and students' 11th grade academic performance. We find that 
standardized assessments, timing of algebra, and course failures in middle school provide useful 
indication of students' high school academic success. Our aim is not to identify any causal 
mechanism by which middle school achievement leads to high school success or failure, but 
rather to describe important associations that may aid policymakers and school leaders to 
develop strategies early in students' educational pursuit of the high school diploma. [PDF file] 


Mac Iver, M., Plank, S.B., Durham, R., Farley- Ripple, E., & Balfanz, R. (2008). The 
challenge of on-time arrival: The seven-year flight paths of Baltimore’s sixth 
graders of 1999-2000. Research Report. Baltimore, MD: Johns Hopkins University. 
http ://baltimore-berc.org/pdfs/SIXTH % 20pathways5-13-08.pdf 

BERC website description: To better understand and address the significant decline in 
achievement or engagement that occurs for many of Baltimore’s middle school students, The 
Challenge of On-Time Arrival study retrospectively followed a cohort of students forward from 
sixth grade to several years past their on-time high school graduation year. It identified 
characteristics of middle grades environments associated with the most encouraging student 
trajectories or those associated with declines in achievement levels, attendance, or educational 
progress. 
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Morris, J.D., Ehren, B.J., & Lenz, B.K. (1991). Building a model to predict which fourth 
through eighth graders will drop out in high school. Journal of Experimental 
Education, 59(2), 286-293. 

Authors’ abstract: The classification accuracies of models for predicting later high school drop- 
out from data available in grades 4 through 8 were examined. The objective was to construct an 
actuarial dropout screening model to serve as part of an overall drop out identification paradigm 
for the state of Florida. Subjects were 503 dropouts and 282 persisters from six geographically 
representative school districts in Florida. Separate superior prediction models were identified for 
each grade level and yielded cross-validated classification accuracies ranging from 73% to 88% 
for the dropout group and 73% to 86% for the persister group. All classification accuracies for 
each grade for both the dropout and persister groups and for the combination of the two were 
significantly greater than chance expectation and were considered of practical importance as 
well. [PDF file] 


Plank, S.B., Durham, R.E., Farley-Ripple, E., & Norman, O. (2008). First grade and 
forward: A seven-year examination within the Baltimore City Public School 
System. Baltimore, MD: Baltimore Education Research Consortium. 
http ://baltimore-berc.org/pdfs/FIRST % 20pathwavs % 205-13-08.pdf 


BERC website description: The First Grade and Forward study analyzed a cohort of students 
who entered first grade in 1999 through their expected (on-time) seventh- grade year to track 
patterns of promotion and retention, attendance, mobility within City Schools, and transfer out of 
City Schools. 


(2010). Early warning: Why reading by the end of third grade matters. The Education 
Digest, 76(1), 27-31. 

Abstract: An article condensed from a summary based on a 2010 KIDS Count Special Report. 
Millions of American children complete third grade without learning to read proficiently. The 
problem is particularly noticeable among low-income children, but reading proficiently by the 
start of fourth grade is a vital indicator in a child's educational development. A lack of 
proficiency is associated with higher rates of school dropout, which limits individual earning 
potential as well as the nation's competitiveness and general productivity. [PDF file] 
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Table 1: States and districts that have existing success factors/early warning databases 


1 


State 

When indicator is 
administered 

Indicators 

Details (excerpted from 
websites or articles) 

Sources 

Alabama 

Pre-K— 12 th grade 

• Attendance (absences; students who 
miss more than 10% of instructional 
time are flagged) 

• Behavior (teacher referrals, 
suspensions, and more) 

• Course Credits (total number of F’s in 
all courses and total number of F’ s in 
core academic courses, especially in the 
area of Language Arts/Reading and 
Math; Two or more F's in core 
academic courses and/or fewer than 
one-fourth of the credits required to 
graduate minus one indicate that a 
student is off track to graduate with 
identified cohort.) 

Local education agencies have the option to 
include additional indicators at their discretion. 

In 2011, ALSDE partnered 
with a software development 
company (STI) to create a 
Graduation Tracking System 
to support the state’s “Every 
Child a Graduate” Dropout 
Prevention Awareness 
Campaign. The system is 
available to all schools, 
districts, and local education 
agencies in the state at no 
additional cost. The system 
provides alerts to counselors, 
graduation coaches, 
administrators, response -to- 
intervention coordinators, 
and other educators as 
needed for students who are 
at risk of school failure. 

Alabama State Department of Education, 
Prevention and Support Services (December 
2009). The Development and 
Implementation Guide for the Alabama 
Graduation Tracking System (AGTS). [file 
attached] 

Alabama Department of Education Partners 
with STI to Provide Graduation Tracking 
System to Schools. 

http://www.cblohm.com/news/STI/STI 1 10 
428/ 

Every Child a Graduate: Executive 
Summary. Alabama Select Commission on 
Fhgh School Graduation and Student 
Dropouts. 

http://media.al.com/bn/other/lTigh%20Scho 

ol%20Dropout%20Commission%200fficial 

%20Report.pdf 


"Civic Enterprises in partnership with the Everyone Graduates Center at Johns Hopkins University will release a report to the nation on early warning and intervention systems 
in summer 2011. This nationwide study is currently underway to evaluate, among other things, the quality and operation of many different systems, the costs and benefits of 
each, and their scalability" ( http://www.americaspromise.org/Our-Work/Grad- 

Nation/~/media/Files/Our%20Work/Grad%20Nation/Building%20a%2QGrad%20Nation/Building%20a%20Grad%20Nation FullReport FINAL.%2011-30-10.ashx . p.45). 
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State 

When indicator is 
administered 

Indicators 

Details (excerpted from 
websites or articles) 

Sources 

Louisiana 

9 th grade 

• Student attendance 

• Student course achievement 

• Student behavior 

• Student age 

“Louisiana is a trailblazer in 
utilizing its early warning 
system, balancing the need to 
report on risk factors 
identified at the state level 
with the flexibility to adjust 
risk factors at the local level. 
At the state level, the system 
flags students who are absent 
at least 10 percent of the days 
they are enrolled, are over-age 
for their grade level, or whose 
grade point averages have 
decreased by at least 0.50. 
Because the specific trigger 
points can vary by district and 
school, both can make 
modifications as needed. To 
ensure that staff can use the 
data to develop and deliver 
effective interventions for 
identified students, the system 
automatically emails results to 
district and school leaders 
twice a month. 

The state also provides 
incentives for districts and 
schools to use this system in 
order to improve outcomes for 
students. Schools can receive 
state grants and technology for 
developing intervention 
programs and producing 
student reports.” 

( http://aDDlication.iff.org/draD 
out/view state. php?id=901). 

Jobs for the Future. Dropout Prevention and 
Recovery: Louisiana. 

httD://application. iff.org/dropout/view state. 
php?id=901 

Pinkus, L. (2008). Using early-warning data 
to improve graduation rates: Closing cracks 
in the education system. Washington, DC: 
Alliance for Excellent Education. 
http://www.all4ed.org/files/EWI.pdf 

Data Quality Campaign. (2009). Louisiana 
Dropout Early Warning System (DEWS). 
http://www.dataqualitvcan 1 paign. 0 rg/files/E 
WI - DEWS -Presentation Webinar .pdf 
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State 

When indicator is 
administered 

Indicators 

Details (excerpted from 
websites or articles) 

Sources 






Maine 

9 th grade 

Academic Performance 

• State Assessment - Math 

• State Assessment - Science 

• State Assessment - Writing 

• State Assessment - Reading 

Educational Engagement 

• Number of out of school suspensions 

• Expulsions 

Student Background 

• Repeated one or more grades 

• Transfers 

• ACCESS for ELLs Score 

• Special Education 

• Free or Reduced Lunch 

• 2 or more years over age for entering 
9 th grade 

At-Risk Data Mart 
A tool that educators can use 
to identify, address, and 
manage a problem, 
opportunity or strategy. 

Maine is using the tool to 
create a dropout early warning 
and management system: 

• Identify students at 
risk 

• Allow educators to 
create, assign, and 
manage programs for 
at-risk students and 
track student 
performance in the 
programs 

• Provide data for 
analysis of model 
results as well as 
analysis to improve 
the model and 
programming 

The At Risk Data Mart 
contains two main areas of 
reporting: 

• Model Roster — a 
quick link with 
Access to all students 
Model Scores 

• Reports — contains 

Hurwitch, W., & Stefanakos, M. (2010). 
Using SLDS data to improve student 
achievement — The Maine Growth Model 
and at-risk students data marts. STATS DC 
2010 NCES Summer Data Conference, 
Bethesda, MD. 

http://nces.ed.eov/whatsnew/conferences/sta 
tsdc/20 10/session V.asp 
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State 

When indicator is 
administered 

Indicators 

Details (excerpted from 
websites or articles) 

Sources 




Data Snapshots and 
Data Tables for 
deeper analysis 


Massachusetts 

9 th grade 

• The student's spring grade 8 MCAS 
mathematics score 

• The student's spring grade 8 MCAS 
English language arts (ELA) score 

• The student's spring grade 8 attendance 
rate 

The Commonwealth's Early 
Warning Indicator Index uses 
statewide data from the most 
recent graduating cohort as the 
starting point for helping local 
school districts identify 
students who may be at risk of 
not graduating on time from 
high school. Incoming high 
school freshmen are assigned 
one of five risk levels (Very 
High Risk, High Risk, At Risk, 
Borderline, or Low Risk) 
based on their middle school 
academic performance and 
attendance record the previous 
school year. Detailed rosters 
for these students — the 
majority of whom are likely to 
be enrolled in grade 9 the 
upcoming school year — are 
provided to the 
Commonwealth’s 24 urban 
school districts in a Microsoft 
Excel workbook via the 
Department’s online Security 
Portal. 

Massachusetts Department of Elementary 
and Secondary Education. (March 2010). 
2009— 10 Early Warning Indicator Index. 
http://www.doe.mass.edu/dropout/fvlOearlv 
indicator.pdf 

Rhode Island 

6 th grade 

• Total number of freshmen who have 
personal literacy plans; 

• Total number of students who have 
failed Algebra I or ninth grade math; 

• Total number of students who are 

Based on HB 5351 (2007), the 
Department of Elementary and 
Secondary Education is 
required to work with the 
school districts that have the 

Deye’, S. (April 2010). Going to scale: 
Working with state legislators to prevent 
and re-engage dropouts. National 
Conference of State Legislatures 
Grad Nation Spring Training: Going to 
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State 

When indicator is 
administered 

Indicators 

Details (excerpted from 
websites or articles) 

Sources 



repeating the ninth grade; and 
• Total number of students receiving 
remedial programming in the ninth 
grade. 

lowest high school graduation 
rates and incorporate 
progressive support and 
intervention with specific 
dropout prevention strategies 
and targeted resources. The 
department is to develop or 
identify specific methods of 
targeted intervention for 
school districts that have a 
dropout rate higher than 15 
percent. 

Scale. (PowerPoint presentation), [file 
attached] 

South Carolina 

8 th — 12 th grade 

PACT Results are displayed for grades 3, 6, 
and 8. Results are listed ‘BELOW BASIC’ are 
flagged as an at-risk characteristic. 

Credits Earned are flagged as at-risk 
indicators if there are <5 for grade nine, <11 for 
grade ten and <17 for grade eleven. 

Years Overage indicates the years a student 
is overage for the grade; >=2 years is flagged as 
an at-risk characteristic. 

Total Discipline Events is the total 
discipline events for a specific enrollment 
period. 

At-Risk Discipline Events is a total of the 
discipline events coded as 150, 151, 152 and = 
(500-743). 

Total Disposition Events is a total of all 
disposition events for a specific enrollment 
period. 

Suspension/Expulsion Events is a total of 
the at-risk disposition events that are coded as 
SUS, SUPX and EXP for a specific enrollment 
period. 

Absence Days is a total number of the days 

SC Department of Education 
has developed the statewide 
Student Potential Performance 
Snapshot early warning 
system. 

Student Potential Performance Snapshot. 
EEDA At-Risk Committee -IT Initiative. 
Data Quality Campaign Webinar. August 
26, 2009. http://dqcampai 2 n.org/f 1 les/EWI - 
DQC SPPS Presentation.pdf 
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State 

When indicator is 
administered 

Indicators 

Details (excerpted from 
websites or articles) 

Sources 



a student was absent during a specific 
enrollment period; >8 absences is flagged as an 
at-risk characteristic. 

Times Retained indicates a student did not 
advance to the next grade; >=1 is flagged as an 
at-risk characteristic. 

Multiple Enrollments if flagged at-risk, 
indicates a student was enrolled in multiple 
schools during the same school year. 

Homeless is an indicator recorded for a 
student if a counselor learns the student is 
without a home. 

Single Parent indicates the student is a 
single parent. 

Displaced Homemaker indicates the 
students’ parent is now in search of a job outside 
the home due to death, divorce or abandonment. 

9thGrade Math is the lowest math grade 
received by the student for a math course for 
credit. If the grade is <=69 then the value is 
flagged as an at-risk characteristic. 

9thGrade English is the lowest English 
grade received by the student for an English 
course for credit. If the grade is <=69 then the 
value is flagged as an at-risk characteristic 



Tennessee 

9 th grade 

• Attendance under 85% 

• Failed two or more courses 

• Suspended two or more times 

Tennessee is currently 
developing a statewide early 
warning system. 

National Governors Association meeting. 
(October 2010). Tennessee Dropout 
Prevention and Recovery Action. 
http://www. n ga. or g/Files/ pdf/ 101 OGRADU 
ATIONTN.PDF 




The Everyone Graduates 
Center at Johns Hopkins 

Balfanz, R., Wang, A., Byrnes, V. (2010). 
Early warning indicator analysis: 
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State 

When indicator is 
administered 

Indicators 

Details (excerpted from 
websites or articles) 

Sources 




University conducted the 
study and developed the 
indicators for the Tennessee 
Department of Education as 
part of its early warning 
system development. 

Tennessee. Baltimore, MD. Johns Hopkins 
University, Everyone Graduates Center. 
http://www.state.tn.us/education/safe schls/ 
dropout/doc/FinalEarl vW arnin glndicator An 
alvsis.PDF 


District 

When indicator is 
administered 

Indicators 

Details 

Sources 

Baltimore City 

6 th grade 

• Chronic absence (defined as missing 20 

Baltimore Education Research 

Baltimore Education Research Consortium. 

Public 


or more days of school) 

Consortium (BERC) is a 

(2011). Destination graduation: Sixth grade 

Schools, MD 

9 th grade 

• Failing English, or math, or both and/or 
a failing average for English, math, 
science, and social studies 

• Being at least one year overage 
(suggesting an earlier retention) 

• Being suspended for three or more days 

• Chronic absenteeism 

• Core course failure 

partnership between Morgan 
State University, Johns 
Hopkins University, & 
Baltimore City Public 
Schools. 

early warning indicators for Baltimore City 
Schools. Their prevalence and impact. 
Baltimore, MD: Author. http://baltimore- 
berc.org/pdfs/SixthGradeEWIFullReport.pd 
f 

Mac Iver, M.A., & Mac Iver, D.J. (2010). 
Keeping on track in ninth grade and 
beyond: Baltimore ’s ninth graders in 2007- 
08. A Research Report by the Baltimore 
Education Research Consortium. 
http://baltimore- 

berc.org/pdfs/3A%20Final%20report 06- 
15-10. pdf. 

Charleston 

All grade levels 

Attendance 

At-Risk Alert System 

Stevens, B. (September 25, 2009/. ARAS 

County School 


Number of missed 

(ARAS): 

Charleston County At-Risk Alert System. 

District, SC 


instruction days 

•Helps identify students 

REL-SE EEP presentation, [file attached] 
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District 

When indicator is 
administered 

Indicators 

Details 

Sources 



Age 

Comparing age to 
current grade level 

Grades 

Passage of current ELA and Math courses 

Lunch 

Student's FRPL meals status 

Suspensions (ISS) 

Total number of incidents 

Suspensions (OSS) Total number of incidents 

PACT/HSAP ELA 

Performance Level 

PACT/HSAP Math 

Performance Level 

potentially at-risk by using 
existing academic & behavior 
data. 

•Transforms data into reports 
to support effective decision 
making. 

•Provides composite views of 
magnitudes of risk factors 
existing for students and 
schools. 

•Supports a variety of student 
support models 

The reports assign a risk level 
for each student based on the 
student’s score on each of the 
8 indicators: Low, Moderate, 
or High Risk. 

Also see: 

Charleston County School District. 
(November 2008). At-Risk Alert System 
(ARAS) 2007-08 Survey Results. Brief No. 
08-047. Department of Assessment and 
Accountability. 

http://www. ccsdschools.com/ReDorts Statis 
tics/documents/AT- 

RISK ALERT SYSTEM ARAS2007- 

08REPORT.pdf 

Charlotte- 
Mecklenburg 
Public 
Schools, NC 

9 th — 12 th grades 

The district developed a dropout prediction 
model based on certain data, including: student 
demographic and achievement information 
(grade, race, SPED, and Gifted status, overage- 
for-grade. Algebra I and English I performance), 
as well as attendance, retention, mobility, and 
withdrawal code information. 

Microsoft Excel rosters were 
created for each school that 
contained pertinent 
information and their 
associated risk. The School 
Social Work office will share 
these rosters with school 
counselors prior to the start of 
the 2008-09 school year. 
Counselors will use these 
rosters and the information 
contained within them to focus 
efforts on those students most 

Charlotte-Mecklenburg School. (2008). 
Research Brief: 2007—08 Dropout 
Prediction Model. Center for Research & 
Evaluation 

http://www.cms. kl2.nc.us/cmsdepartments/ 

accountabilit v/ cfre/Documents/2007 - 
08%20Dropout%20Prediction %20Model . pd 
f 
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administered 

Indicators 

Details 

Sources 




in need in the hopes of 
keeping students engaged in 
school. 


Chicago Public 
Schools, IL 

9“ grade 

First-time freshman students are considered On- 
Track at the end of their freshman year if they 
have accumulated at least five course credits and 
failed no more than one semester course in a 
core subject (English, math, social science, or 
science) during the school year. 

Publicly available on-track 
rates for every CPS school 
from 1997-2010, at 
httr>://research.cr>s.kl2.il.us/cD 
s/accountweb/Rer>orts/allscho 
ols.html. 

High School Score Card 

httr>://research.cr>s.kl2.il.us/cr>s/accountweb 

/ReDorts/allschools.html 

On-Track Fact Sheet [file attached] 

Allensworth, E., & Easton. J. (2007). What 
matters for staying on-track and graduating 
in Chicago public high schools: A close 
look at course grades, failures and 
attendance in the freshman year. Chicago: 
Consortium on Chicago School Research. 
httn://ccsr. uchicaeo.edu/nublications/07%20 
What%20Matters%20Final.Ddf 

Ponder, P. (n.d.) Graduation pathways: 
using freshman year indicators. Doing What 
Works. 

http://dww.ed. gcv/Dronout- 
Prevention/Rigorous-Relevant- 
Instruction/see/index.cfm?T ID=24&P ID= 

53&cl=1254 

Colorado 
(5 LEAs) 

9 th grade 

• Poor attendance 

• Behavior problems 

• Course failure 

A project of the Colorado 
Graduates Initiative, this 
partnership with Johns 
Hopkins University works 
with five of Colorado’s school 
districts to help them 
understand both the behavioral 
patterns of recent dropouts in 
the years prior to leaving 

Mac Iver, M.A., Balfanz, R., & Byrnes, V. 
(October 2009). Advancing the “Colorado 
Graduates ” agenda: Understanding the 
Dropout Problem and Mobilizing to Meet 
the Graduation Challenge. The Center for 
Social Organization of Schools Johns 
Hopkins University. 

httt>://www. schoolenaagement.org/truancvt) 
reventionregistrv/admin/Resources/Resourc 
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District 

When indicator is 
administered 

Indicators 

Details 

Sources 




school, and who is currently at 
risk of dropping out of school 
(by looking at data from 
today’ s students in earlier 
grades). 

es/ Ad van ci n nth e. nd f 

Mac Iver, M.A., Balfanz, R., & Byrnes, V. 
(April 2009). Dropouts in the Denver Public 
Schools: Early warning signals and 
possibilities for prevention and recovery. 
The Center for Social Organization of 
Schools, Johns Hopkins University. 
http://extras.mnginteractive.com/live/media/ 
site36/2009/05 15/200905 15 122610 Denve 

r Dropout Report Final.pdf 

Dallas 
Independent 
School 
District, TX 

9 th grade 

• The student has accumulated five full 
course credits 

• The student has no more than one 
semester F (that is, one-half of a full 
credit) in a core subject (English, math, 
science, or social studies). 

Uses the Consortium on 
Chicago School Research’s 
(CCSR) on-track indicators. 

Weir, D. (2008). Development of the Dallas 
ISD on-track indicator , 2007-08. 
http://www.dallasisd.org/eval/evaluation/atg 
Iance2009/EA08- 153 -4- At-a-Glance-On- 

Track-Indicator, pdf 

Los Angeles 
Unified School 
District, CA 

High school: 

CST English/ Language 
Arts (yearly) 

CST Mathematics (yearly) 
Attendance (monthly) 
Periodic Assessment Data 
Mathematics (quarterly) 
Language Arts ( quarterly) 
Grade Point Average (by 
grading period) 

Work Habits (by grading 
period) 

Middle school 

High School indicators: 

• Age minus grade index ( students at 
grade level, average value = 6) 

• Attendance rate 

• Cumulative grade point average 

• Number of Unsatisfactory” Marks in 
Work Effort 

• Number of Fails on Achievement 
Marks 

Middle School indicators: 

• Creates a flag when students fall below 
a certain threshold: 

• Far Below Basic in CST in ELA or 
Math 

• Two or more “F” marks in last 

LAUSD’s “Decision Support 
System.” 

Lim, C., & Pirone, J. (2007). Using data for 
dropout prevention: can we identify at risk 
students before it’s too late? Planning and 
Assessment, Los Angeles Unified School 
District. 

http://www.mvliituremvdecision.org/ourpag 
es/auto/2008/5/ 1/1209652805638/Data%20 
Driven%20Decision%20Making.ppt#l 

Silver, D., Saunders, M., & Zarate, E. 
(2008). What factors predict high school 
graduation in the Los Angeles Unified 
School District ? Santa Barbara: California 
Dropout Research Project, [file attached] 
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semester 

• Cumulative GPA below 1.5 

• Far Below Basic in latest ELA or Math 
Periodic Assessment 

• Attendance rate below 90% 

• One or more suspension incidences 



New York City 
(63+ schools) 

9 th -12 th grade 

Uses New York State’s graduation requirements 
(a combination of course credits and pass grades 
in a series of end-of-course state Regents exams) 
to map out the progress students need to make 
each year to graduate with a regular diploma in 
four years. In addition, the metric includes 
indicators of college readiness that go beyond 
the state’s high school requirements. 

Developed by New Visions 
for Public Schools, an 
education reform organization 
in New York City (NYC) that 
supports more than sixty-three 
of the city’s public schools, 
and based partially on CCSR 
research and findings from 
NYC’s Department of 
Education’s Office of Multiple 
Pathways. 

Using this metric, each 
student’s status is described 
through the use of color-coded 
categories: on track to college 
readiness (blue); on track to 
graduation (green); almost on 
track to graduation (yellow); 
or off track to graduation 
(red). New Visions analyzes 
the data for each school and 
provides administrators with a 
school-wide report on student 
performance based on the 
metric. 

Pinkus, L. (2008). Using early-warning data 
to improve graduation rates: Closing cracks 
in the education system. Washington, DC: 
Alliance for Excellent Education. 
http://www.all4ed.or2/files/EWI.pdf 

Philadelphia, 

6 th -8 th grade 

• Attending less than 80% of the time 

Based on research conducted 

Herzog, L. (2009, Spring). Two schools use 
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District 

When indicator is 
administered 

Indicators 

Details 

Sources 

PA (2 schools) 


• Receiving a poor final behavior mark in 
a course 

• Failing Math 

• Failing English 

by Robert Balfanz, co-director 
of the Center of Social 
Organization of Schools at 
Johns Flopkins University, and 
the Philadelphia Education 
Fund, the Diplomas Now 2 
project applies the four early 
warning indicators identified 
by Johns Hopkins 
researchers — poor attendance, 
repeated behavior problems, 
failing English, and failing 
math — to target 6 th -8 th grade 
students in need of additional 
support, then uses City Year 
and Community in Schools 
workers to provide the 
interventions to keep the 
students on track for 
graduation. 

‘early warning system’ to avert dropouts. 
Philadelphia Public School Notebook, 

76(3), 26-27. 

http://www. thenotebook. or g/ sprin g- 
2009/ 091091/ two-school s-use- 
%E2%80%98earlv-warning%E2%80%99- 
svstem-avert-dropouts 

http://diplomasnow.org/results/philadelphia/ 

Neild, R.C., & Balfanz, R. Unfulfilled 
promise: The dimensions and 
characteristics of Philadelphia’ s dropout 
crisis, 2000-2005 (Baltimore: Center for 
Social Organization of Schools, Johns 
Hopkins University, 2006). [file attached] 

Balfanz, R. (2008). Early warning and 
intervention systems: Promise and 
challenges for policy and practice. Prepared 
for National Academy of Education and 
National Research Council Workshop on 
Improved Measurement of High School 
Dropout and Completion Rates. 
http://www7.nationalacademies.org/BOTA/ 
Paper%20bv%20R.%20Balfanz.pdf 


"Diplomas Now, a school turnaround model, works with 10 school districts including Chicago, Los Angeles, New Orleans, Detroit, Washington, D.C., and San Antonio on building 
and enhancing early warning indicators and deploying the resources of City Year, a national service organization dedicated to reducing high school dropout. Communities in 
Schools, a youth serving organization that provides high-needs students with community supports to complete their educations, and Talent Development, which reforms 
curriculum, instruction, school climate, and school leadership, to ensure that students get all of the supports they need to graduate from high school. Early results in multiple 
sites show significant reductions in absences and behavior problems and increases in student academic achievement in reading and math" 
( http://www.americaspromise.org/Our-Work/Grad- 

Nation/~/media/Files/Our%20Work/Grad%20Nation/Building%20a%20Grad%20Nation/Building%20a%20Grad%2QNation FullReport FINAL%2011-30-10.ashx . p. 45). 
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Balfanz, R. (2009). Putting middle grades 
students on the graduation path. A policy 
and practice brief. Everyone Graduates 
Center and Talent Development Middle 
Grades Program. 

http://www.nmsa.or e/portals/O/pdf/research/ 

Research from the Field/Policv Brief Bal 

fanz.pdf 

Portland Public 
Schools, OR 

1 st , 3 rd , 6 th -8 th , 10 th , & 12 th 
grades 

Enter 1st Grade: Ready to Read 

• Students recognize letter names and 
sounds and parts of words. 

End of 3rd Grade: Reading to Learn 

• Students are able to exceed the 
benchmark on the state reading test. 

Middle Years: Ready for High School 

• Student attendance is 90%or greater. 

• Students meet the benchmark on the 
state writing test. 

• Students take and pass 8 th grade 
algebra. 

Enter 10th Grade: On Track to Graduate 

• Students complete 9th grade with 6 
credits and C grade in core subjects 

12th Grade: Graduating on Time Ready for 

Work & College 

• Students graduate on time. 

• Students meet college-ready 
benchmark on at least 3ACT tests. 

Portland Public Schools’ 
Milestones Framework 

Portland Public Schools. Measuring 
Results: Tlie Milestones Framework. 
http://www.pps.kl2.or.us/departments/miles 
tones/1 777.htm 

Portland Public Schools (September 2010). 
2009-10 Milestones Update. Student 
Achievement Committee 
http://www.pps. kl2.or.us/files/milestones/M 
ilestonesUpdate.pdf 

Cielo, M.B., & Leveen, L. (2007). The 
fourth R: New research shows which 
academic indicators are the best predictors 
of high school graduation — and what 
interventions can help more kids graduate. 
Portland, OR: Connected by 25. 
www.connectedbv25.ore/file download/46 
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The Regional Educational Laboratories have been involved in several efforts to research the 
effectiveness of various early warning indicators. See: 

Uekawa , K., Merola, S., Fernandez, F., & Porowski, A. (January 2010). Creating an 
early warning system: Predictors of dropout in Delaware. Regional Educational Laboratory 
Mid-Atlantic. 

http://www.doe.kl2.de.us/infosuites/ddoe/p20council/docs/MA1275TAFINAL508.pdf 

Hartman, J., Wilkins, C., Gregory, L., Gould, L.F., & D’Souza, S. (2011). Applying an 
ontrack indicator for high school graduation: adapting the Consortium on Chicago School 
Research indicator for five Texas districts. (Issues & Answers Report, REL 2011-No. 100). 
Washington, DC: U.S. Department of Education, Institute of Education Sciences, National 
Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory 
Southwest. http://ies.ed.gov/ncee/edlabs/regions/southwest/pdf/REL 201 1 100.pdf 

Also, Regional Educational Laboratory Southeast is currently conducting similar work in 
Broward County, EL, at the request of Broward County Public Schools (Fast Response 1.2.50: 
Early Indicators of High School Student Failure in a Large Urban District in Florida ). 
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REGIONAL EDUCATIONAL LAB August 2011, EBE # 833A 




We provide research based information on 
educational initiatives happening nationally and 
regionally. The EBE Request Desk is currently taking 
requests for: 

- Research on a particular topic 

- Information on the evidence base for curriculum 
interventions or professional development 
programs 

- Information on large, sponsored research projects 

- Information on southeastern state policies and 
programs 

For more information or to make a request, contact: 
Karla Lewis 
1.800.755.3277 
klewis@serve.org 


The Regional Educational Laboratory (REL) - Southeast's Evidence Based Education (EBE) Request Desk is a service provided by a 
collaborative of the REL program, funded by the U.S. Department of Education's Institute of Education Sciences (IES). This response 
was prepared under a contract with IES, Contract ED-06-CO-0028, by REL-Southeast administered by the SERVE Center at the 
University of North Carolina at Greensboro. The content of the response does not necessarily reflect the views or policies of IES or the 
U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the 
U.S. Government. 

Not verified as meeting IES standards; not for distribution. 
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